Skip to content

Rebuild as a Python-native framework for Apache Dubbo - #53

Draft
cnzakii wants to merge 1 commit into
apache:mainfrom
cnzakii:python-native-rebuild
Draft

cnzakii wants to merge 1 commit into
apache:mainfrom
cnzakii:python-native-rebuild

Conversation

@cnzakii

@cnzakii cnzakii commented Nov 15, 2026 •

Copy link
Copy Markdown
Member

Fixes apache/dubbo#16595

Summary

This PR replaces the previous implementation with a new Python-native framework for Apache Dubbo. It runs on asyncio, with AnyIO for structured concurrency and uvloop as an option, and supports Python 3.11 to 3.14 on Linux, macOS and Windows.

It talks to Dubbo Java and Dubbo Go over both the Triple and Dubbo protocols, finds and registers services through ZooKeeper and Nacos as they do, and brings the governance of the other Dubbo languages to Python.

Highlights

  • Works with Dubbo Java and Go. Triple with Protobuf and Dubbo with Hessian 2, ZooKeeper and Nacos registries, and both application-level and interface-level service discovery. Wire formats and registry entries follow Dubbo Java and Dubbo Go.
  • Compatible with gRPC. Triple runs over HTTP/2, so gRPC clients and servers, grpcurl, health checks and reflection work with it.
  • Pythonic. A service is a typing.Protocol with type hints, calls are async, and an application runs inside async with, in any asyncio program. Each method of a reference has the name, signature, type annotations and docstring of its contract, so that tools that read them, such as AI agent frameworks, can take it as a function.
  • Governance built in. Load balancing, cluster fault tolerance, routing rules and dynamic configuration, as in the other Dubbo languages.
  • Extensible. Every protocol, registry, load balancer, router and filter is a plugin that a package can add through an entry point.
  • Measured. Benchmarks against Dubbo Java, Dubbo Go and the gRPC libraries of Python, and several rounds of tests at scale with fault injection, below.

Features

  • Protocols
    • Triple over HTTP/2, interoperable with gRPC: unary, server-streaming, client-streaming and bidirectional calls; deadlines, rich error details, and gzip, deflate and bzip2 compression; the gRPC health and reflection services; the wrapped mode for Java interfaces.
    • The Dubbo protocol with Hessian 2.
    • TLS and mutual TLS on both protocols.
  • Services and calls
    • Contracts as typing.Protocol, or generated from .proto files by protoc-gen-python-dubbo.
    • A mapping of Java types, and generic calls without the contract.
    • Attachments and a call context that propagate along a chain of calls.
  • Service discovery
    • Application-level and interface-level registration on ZooKeeper and Nacos.
    • MetadataService 1.0.0 and V2, local and remote metadata, and metadata reports.
    • Configuration centers on ZooKeeper and Nacos.
  • Traffic management
    • Clusters: failover, failfast, failsafe, failback, forking, broadcast, available and zone-aware.
    • Load balancers: random, round-robin, least-active and consistent-hash.
    • Condition and tag routers, and dynamic configurators.
    • Filters for access logs, active-call and concurrency limits, TPS limits, tokens and deprecated methods.
  • Application
    • Configuration from code, from JSON, YAML and properties files, and from environment variables, in layers.
    • Lifetimes that open and close as a whole, and graceful shutdown within a time budget.
  • Packaging
    • The default installation serves and calls over both protocols. Extras add zookeeper, nacos, yaml, properties, uvloop and protoc.

Samples

samples/ holds 22 samples, each with a README. The tests run them as their READMEs say. CI skips the logging, extensions, TLS, FastAPI and Pydantic AI samples, whose packages it does not install.

  • Basics: 17 samples, from a first call through registries, configuration, streaming, errors and timeouts, routing, filters, extensions, TLS and generic calls.
  • Interoperability: Python and Dubbo Java calling each other through Nacos over Triple and over Dubbo; Python and Dubbo Go over Triple; gRPC clients and servers.
  • Integrations:
    • A small shop behind a FastAPI gateway, with a trace id along the chain of services.
    • An AI agent built with Pydantic AI, which plans restocking with Dubbo services as its tools, with any model served through the Chat Completions API of OpenAI.

Performance

benchmark/ measures each server with the same echo service, Protobuf over Triple and Hessian 2 over Dubbo, under the same load: on Linux with a 32-core Xeon, the server pinned to its cores with taskset and the load on the others, the median of three rounds. None of the 81 million calls failed.

Calls per second, with 128-byte / 16 KiB payloads:

Server 1 core, Triple 1 core, Dubbo 8 cores, Triple 8 cores, Dubbo
Dubbo Python 13.5k / 10.8k 22.2k / 17.4k 13.7k / 10.7k 21.4k / 17.0k
Dubbo Java 5.1k / 3.1k 35.0k / 6.4k 85.6k / 50.4k 309k / 61.3k
Dubbo Go 6.8k / 2.5k 21.4k / 6.4k 45.0k / 17.9k 104k / 18.8k
grpcio 3.5k / 3.1k
grpclib 4.1k / 3.0k
  • On one core, Dubbo Python serves the most Triple calls of the servers measured, and over Dubbo about as many as Dubbo Go.
  • A Python process serves on one core, while Java and Go use every core they are given, so on hosts with several cores they serve more per process.

Tests at scale

Besides CI, the framework went through several rounds of tests at scale on Linux hosts, with providers and consumers in containers beside Dubbo Java, Dubbo Go, ZooKeeper and Nacos:

  • providers killed while they are called;
  • 50 providers with 50 consumers, and 100 services over 300 instances;
  • rolling releases and registry outages;
  • weights changed while calls run;
  • long runs;
  • the benchmarks above.

Compatibility

  • Public API: the API is new, so code written for earlier releases needs to be ported.
  • Wire formats and interoperability: wire formats and registry entries follow Dubbo Java and Dubbo Go, with golden files captured from them in the tests. The samples call Dubbo Java 3.3 over both protocols and Dubbo Go over Triple, through Nacos, and gRPC clients and servers.

Testing

  • Tests: 1,591 unit and integration tests, with 96% line and branch coverage, without the generated code. The ZooKeeper and Nacos tests run against servers in Docker.
  • CI:
    • Ruff, ty and the license header check.
    • Tests on Linux, macOS and Windows with Python 3.11 to 3.14.
    • Fresh installs of the default installation.
    • Integration tests and the samples against ZooKeeper 3.8 and 3.9 and Nacos 2.4 and 2.5.
    • A package build check.
  • Time: the asynchronous tests run on a virtual clock that jumps to the next timer while every task waits. Deadlines, timeouts and grace periods are asserted exactly, whatever the speed of the host.

@cnzakii
cnzakii force-pushed the python-native-rebuild branch 9 times, most recently from a20307e to 1121c12 Compare November 16, 2026 12:37
@cnzakii
cnzakii requested a review from AlbumenJ November 17, 2026 03:16
@cnzakii
cnzakii force-pushed the python-native-rebuild branch 11 times, most recently from f9e9ad1 to 60b148b Compare November 20, 2026 10:34
Replace the previous implementation with a new one built on asyncio and
AnyIO, with uvloop optional, for Python 3.11 to 3.14:

- the Triple protocol over HTTP/2, interoperable with gRPC, and the Dubbo
  protocol with Hessian 2, both in the default installation;
- application-level and interface-level service discovery on ZooKeeper
  and Nacos, with metadata services and reports, and configuration
  centers;
- clusters, load balancers, condition and tag routers, configurators and
  filters, as extension points loaded by entry points;
- configuration from code, files and environment variables;
- the protoc plugin that generates service stubs;
- samples, from a first call to a small shop of services behind
  FastAPI, an agent of Pydantic AI whose tools are services, and calls to
  and from Dubbo Java and Dubbo Go, each with a README and run by the
  tests;
- end-to-end benchmarks of servers on one core and on eight, beside Dubbo
  Java, Dubbo Go and the gRPC libraries of Python, and of a Dubbo Python
  consumer;
- unit and integration tests on a virtual clock, CI on GitHub Actions,
  and release candidates for the release manager to sign.

Extras add the ZooKeeper and Nacos clients, the YAML and properties
configuration formats, uvloop and protoc.
@cnzakii
cnzakii force-pushed the python-native-rebuild branch from 60b148b to 6510188 Compare November 20, 2026 11:53
@cnzakii
cnzakii marked this pull request as ready for review November 20, 2026 12:12
@cnzakii
cnzakii removed the request for review from AlbumenJ November 20, 2026 13:26
@cnzakii
cnzakii marked this pull request as draft November 22, 2026 15:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] [Dubbo-python] Rebuild Dubbo Python as a Python-native framework

1 participant